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Introduction 



The ground field Ik is algebraically closed and of characteristic zero. Let g be a reductive 
algebraic Lie algebra over k and a an involutory automorphism of g. Then g = g © Qi is 
the direct sum of cx-eigenspaces. Here go is a reductive subalgebra and gi is a go-module. 
Let G be the adjoint group of g and G C G & connected subgroup with LieG = go- The 
commuting variety of (g,go) is the following set: 

= {(x,y) e 0i x Qx | [x,y] = 0}. 

The problem whether <£(fli) is irreducible was considered by Panyushev [H|, [Zj and Sabourin- 
Yu 0, [HI]. Suppose g is simple. Then the known results are 

• if the rank of the symmetric pair (g, g ) is equal to the semisimple rank of g (called 
the maximal rank case), then the corresponding commuting variety is irreducible, [§]; 

• if the rank of (g,go) equals 1, then <£(gi) is irreducible only in one case, namely, 

• for (s[ 2n ,sp 2 n) an d (EqjF^) the corresponding commuting variety is irreducible, [7j; 

• if (g, O ) = {so 2+ m,so 2 ®so m ), then £(qi) is irreducible, [JU|. 

For all other symmetric pairs the problem is open. In sections 1-3, we extend the result of 
[TU] to all symmetric pairs (so n+m ,so n © so m ). The scheme of the proof is similar to that 
of PO]- But as it often happens, the argument in a general situation is shorter and simpler, 
than in a particular case. 

In it was conjectured that (£(fli) is irreducible if the rank of the symmetric pair is 
greater than 1. This conjecture is not true. In section 4, we prove that £(Qi) is reducible for 
symmetric pairs (gl n+m , flt n ©flD wrtn n^rn, (so 2 „,gLj with odd n, and (E e , S0i ©k). 

1. Commuting variety 

The commuting variety <£(g) = {(x, y) G g + g | [x,y] — 0} of a reductive Lie algebra g 
was considered by Richardson in jS], where he shows that £(g) coincides with the closure of 
Cr(txt) for any Cartan subalgebra t C g, and is therefore irreducible. It is not yet known 
whether C(g) is normal or whether the ideal of £(g) is generated by quadrics. 

Let (g, go) be a symmetric pair and c C gi a maximal abelian subspace consisting of 
semisimple elements. Any such subspace is called a Cartan subspace of 0i. All Cartan 
subspaces are G -conjugate, see 0. The dimension of c is called the rank of the symmetric 



pair (g, g ). It is well-known that £o = Go(cxc) is the unique irreducible component of £(gi) 
of maximal dimension, see [UJ Sect. 3]. Here dimtC^!) = dimgx + dime. It follows that 
) is irreducible if and only if = <£o 

The irreducibility problem for £(gi) was first considered by Panyushev in [6 . As was 
mentioned above, £(gi) is not always irreducible. On the other hand, in some particular 
cases one can say more about properties of <£(gi). If (fl,0o) is a symmetric pair of maximal 
rank, then £(gi) is a normal complete intersection and the ideal of £(gi) in k[g! x g x ] is 
generated by quadrics, see [Hj. 

Let n be a non-negative integer. The set 

gS n) = {£ e gi | dimG £ = n} 

is locally closed. Irreducible components of g[ n ^ are called Go-sheets of gi. The following 
statement is well-known for the specialists, but we include a proof here for the sake of 
completeness. 

Lemma 1. Let S be a Go-sheet of gi containing semisimple elements. Suppose for each 
semisimple h & S we have ({h}x (qi)h) C €q. Then ({x}x(q 1 ) x ) c (Co for each x G S. 

Proof. Let igS. Since S contains semisimple elements, they form a dense subset. Therefore, 
we can find a morphism 7 : k — > S such that 7(0) = x and ^(t) is semisimple for each t 7^ 0. 
Then (qi) x = lim^o {di)^{t), where the limit is taken in an appropriate Grassmannian. For 
each y G (qi) x , we can define elements y(t) G (gi) 7 (t) such that y = \im t ^oy(t). Since 
(x, y) = \im t ^o (•y(t),y(t)) and (j(t),y(t)) G €0 f° r eac h t 7^ 0, we conclude that (x,y) G 
U □ 

2. Semisimple and nilpotent elements in q 1 

In this section g = so n+m , g = so n ©£O m . Let V = k n+m be a vector space of the defining 
representation of g. Then we have a Go-invariant decomposition V = V a (B Vb, where V a = k", 
Vb = k m , and gi = k"®k m as a Go-module. Denote by ( , ) the non-degenerate symmetric 
g-invariant bilinear form on V. 

Let Qh be the centraliser of an element h G Q±. Then o induces the symmetric decom- 
position (g^) = (go)h © (gi)/i, where (go)h is the centraliser of h in g . First we describe 
centralisers Qh and (go)/i of semisimple elements h G gi. 

Lemma 2. Let /i G gi fre a semisimple element. Then the symmetric pair (g^,, (go)/ t ) is a 
direct sum (0- =1 (g^,so fc J) © (so n+m _ 2 fc, so n - k ®so m - k ), where k = 

Proof. Recall several well-known facts about semisimple elements of so n+m . Let v x G V be 
an eigenvector of h such that h ■ v x = Xv x and A / 0. Since h preserves the symmetric form 
( , ), we have (v\,V\) = 0. Also if h ■ v — Xv, h ■ w = fiw, then (v,w) 7^ only if A = — 
Let {±Aj, I i — 1, . . . , r} be the set of the eigenvalues of h. Then there is an orthogonal 
/i-invariant decomposition 

V = (V Xl ®V. Xl ) © ... © (V Xr ®V_ Xr ) © v . 
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Here each V\ i is an isotropic subspace, (V\ i ,V±\ j ) = if A, 7^ ±Aj and (Vo,V±x i ) = for 
each Aj. Therefore 0^ C (0^ =1 so(V\ 1 ©^-aJ) © So(Vo). More precisely, if dimVA 4 = h and 
k = tnen 5h = (0i 01*,) © SO n+m _ 2 fc- 

Now it remains to describe (go) ft — (fl/i)' 7 - We may assume that cr is a conjugation by 
a diagonal matrix A G O n+m such that A|y a = —id and A|v 6 = id. Since a{h) = —h, we 
have A ■ V\ i = V-\ t and A ■ Vq — Vq. Moreover, A determines a non-degenerate symmetric 
form ( , )a, on each V\ i by the formula (u, ii;)^ = (v,A- w). Therefore, each soiVx^V-xJ 
is a-invariant, (so(V\ i ®V-\ z )) CT = so^Qso^, and (gl k J a = so^. Finally, the restriction A\ Vo 
has signature (n — k,m — fc). Thus (so n+m _2fc) CT = 50 n _fc©so m _fc. □ 

Denote by the nullcone of Go : 01, i-e., the set of all nilpotent elements in 0i. 

Recall several standard facts concerning nilpotent elements in &t(V). Suppose e G gl(V) 
is nilpotent and m = dimKer(e). Then by the theory of Jordan normal form, there are 
vectors wi, . . . ,w m G V and non-negative integers di,...,d m such that e di+1 -Wi = and 
{e s -Wi 1 < i < m, 0<s< di] is a basis for V. Let C V be a linear span of 
{wi,e-Wi, . . . ,e dl -Wi}. Then the spaces {Vi} are called the Jordan (or cyclic) spaces of the 
nilpotent element e and V = ©^V;. 

Lemma 3. Suppose e G 91(01). Then the cyclic vectors {wi}^ and hence the cyclic spaces 
{Vi} 's can be chosen such that the following properties are satisfied: 

(i) there is an involution i 1— > i* on the set {1, . . . , m} such that di = di*, % — i* if and 
only if dim Vi is odd, and (Vi, Vf) = if i 7^ j* ; 

(ii) o~(wi) = ±Wi. 

Proof. Part (i) is a standard property of the nilpotent orbits in so(V), see, for example, pi 
Sect. 5.1] or j2j Sect. 1]. Then part (ii) says that in the presence of the involution a cyclic 
vectors for e G 9t(gi) can be chosen to be a-eigenvectors, see jH Prop. 2]. □ 

For each e G 9t(gi) we choose cyclic vectors {wi} as prescribed by Lemma El Say that 
e s -Wi has type a if a(e s -Wi) = —e s -w il i.e., e s -Wi G V^; and e s -Wj has type b if e s -Wj G 
Since cr(e) = — e, if e s -Wj G V^, then e s+1 -Wi G and vice versa. Therefore each string 
(wi, e-Wi, . . . , e di -Wi) has one of the following types: 

aba . . .ab, bob . . .ba, aba . . .ba, bob . . .ab. 

Let e G 9t(gi). There is an s^-triple {e, /, /i} such that / G 0i and h G 0o- Recall that e 
is called ei>en if the eigenvalues of ad(/i) on are even. An element e G 9t(gi) is said to be 
a -distinguished (in other notations p- or (— l)-distinguished) if (0i) e contains no semisimple 
elements of [0, 0]. 

Lemma 4. In case (0,0o) = (so n+m ,so n ®so m ) each a -distinguished element e G 9t(0i) is 
even. 

Proof. Let {Vi} be the cyclic spaces of e G 9t(gi) chosen as prescribed by LemmaOl Suppose 
there is an even- dimensional V,. According to [3J Prop. 2], ii Vi has type aba . . .ab, then Vi* 

has type bab . . .ba, i.e., if a(wi) = —Wi, then o~(wi*) = Wi*. Let I be a Levi subalgebra of 
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(so(Vi®Vi*)) e . We may assume that 1 is a-invariant. Then I = sl(k.Wi®k.Wi*) = s^. The 
restriction of a defines a symmetric decomposition [ = [ © ti, where [ = P = SO2. Therefore 
h = I H gi contains semisimple elements. This means that e is not cx-distinguished. Hence, 
all Vi are odd- dimensional and e is even. □ 

3. IS IRREDUCIBLE IN CASE OF (sO n+m , SO n ©SO m ) 

In this section we prove that C(fli) is irreducible in case (g,0o) = (so„ +m ,so n ©so m ). 
The following lemma is taken from ^U|, but the proof given here is shorter. Note that this 
lemma is valid for any symmetric pair (q,Qo)- 

Lemma 5. Suppose e G ^(fli) is even. Then e belongs to a Go-sheet containing semisimple 
elements. 

Proof. Let (e,f,h) be an sl 2 -triple such that / G 0i, h G go- Since e is even, we have 
dimjjh = dimg e . Set e{t) := e — t 2 f for £ G Ik. If t ^ 0, then e(i) is semisimple and 
conjugated to th. Therefore dimg e (t) = dimg^ = dimg e . Clearly e(0) = e = lim 4 ^ e(t) and 
the Go-sheet containing e contains also semisimple elements e(t). □ 

Suppose (g, fl ) — ( s °n+m, so n ©so m ) and let c C 0i be a Cartan subspace. 

Theorem 1. The commuting variety «s irreducible. 

Proof. Recall that Co = Go(cxc). Following the original proof of Richardson jH] (see also [3 
Sect. 2]), we show by induction on dim c that = <£o- The base of induction is the rank 1 

case (so n+ i,so n ), where the irreducibility of <£($ji) is proved in jjj, [Hj. Let (x, y) G £(0i). 

(1) Suppose there is a semisimple element h G 51 such that = [/i, y] = 0. This 
assumption is automatically satisfied if either x or y is semisimple. Moreover, if x (or y) is 
not nilpotent and x = x s + x n is the Jordan decomposition, then x s G 0i and [x s ,x] = 0, 
[x s , y] = 0. 

Consider the symmetric pair (g h , (0o)/i)- Replacing c by a conjugated Cartan subspace, 
we may assume that h G c. Then c is a Cartan subspace of (gi)h- Also, x, y G (gi)^ by 
the assumption. By Lemma0 (Sh,(do)h) = (©[=1(0^^^)) © (so n+ m-2k,SO n -k®SO m -k)- 
Note that each (fl[ fc ., S0fcj) is a symmetric pair of maximal rank, hence, the corresponding 
commuting variety is irreducible, see [01 (3.5) (1)]. Clearly, the commuting variety corre- 
sponding to a direct sum of symmetric pairs is a direct product of the commuting varieties 
corresponding to the summands. Therefore, using the inductive hypothesis, we conclude 
that £((gi)h) is irreducible. Thus (x, y) G (Go)fc(cxc) and, hence, (x, y) G Co- 

(2) It remains to consider pairs of commuting nilpotent elements. Suppose first that there 
is a semisimple element h G Qi such that [x, h] = 0. Then (x, (1 — t)y + th) G £(0i) for each 
t G k and (1 — t)y + th is nilpotent only for a finite number of fs. Therefore, by part (1), one 
has (x, (1 — t)y + th) G £0 f° r almost all t. Since y = lim t _ >0 (l — t)y + th, we get (x, y) G £o- 

(3) Now we may assume that both x and y are a- distinguished nilpotent elements. Ac- 
cording to Lemma x is even. Then, by Lemma x belongs to a Go-sheet containing 
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semisimple elements. According to part (1), the assumptions of Lemma ^ are satisfied and 
it follows that (x, y) G £o- D 



4. Several new examples of reducible commuting varieties 

Let g = g(— 1) © g(0) © g( — 1) be a short grading of a Lie algebra g. Set g := fl(0). Then 
(g,go) is a symmetric pair with gi = g(— l)©g(l). Let c C gi be a Cartan subspace and 
c(— 1), c(l) the images of c under projections on g(— 1) and g(l), respectively. 

Lemma 6. Suppose C(fli) is irreducible. Then 



(4k) G (c(l)xc(l)) = g(l)xg(l). 

Proof. Since each g(— 1) and g(l) consists of nilpotent elements, we have dimc(±l) = dime 
and c = diag(c(-l)ffic(l)). Clearly g(l)xg(l) c £(gi). Since <£(fli) = £ = t7 (cxc), we get 
g(l)xg(l) = g(l)xg(l) n £ = G (c(l)xc(l)). □ 

Corollary. If condition (4k) 7S not satisfied, then £(gi) aas at /east taree irreducible compo- 
nents. 

Now we give three examples of symmetric pairs arising from short gradings such that 
condition (4k) is not satisfied for them. 

Example 1. Suppose (g, go) = (gt n+m , g[„ffig[ m )- Let V be a (n+m)-dimensional vector space 
such that g = gl(V). Let V = 14 © V& be the g -invariant decomposition with dim\4 = n, 
dimV;, = m. Then the involution a is induced by a diagonal matrix A G GL(V) such 
that A\ Va = —id, A\ Vb = id. We have g(l) = B.om(V a ,V b ), g(— 1) = Hom(V r fe , V a ) and 
gi = g(l) © g( — 1). Assume that n < m. Suppose £,77 G gi and [£,77] =0. If £ = X + Y, 
r] = Z + U, where X, Z G Hom(V a , V b ), V, ^ e Hom(V&, V a ), then we set L>i(£,7/) := (X\Z), 
where (X\Z) is an mx2n matrix. 

Now suppose n 7^ m, i.e., n < m. We show that in this case condition (4k) is not 
satisfied. Let c = {X + X* \ X = {x^j),x^j = if i 7^ j} C gi be a Cartan subspace. 
Then c(l) = {X = (xij) \ x^j = if i 7^ j}. It is clear, that rkDi(t,h) < n for each 
pair (t, /i) G cxc. Let g G C7 - Then g = B x C, where 5 G GL n = GL(K), C G 
GL m = GL(V 6 ). If £ = X + Y G gi, then Ad(a)-£ = CXfi- 1 + BYC" 1 . Therefore, 
^•L>i(£,77) := ^i(Ad(a)-£, Ad(a)-7/) = CD x {^r,)B, where 

fi- 1 

fi- 1 



5 



is a non-degenerate 2nx2n matrix. 

Since TkDi(Ad(g)-£,Ad(g)-rj) = rkDi(£,77), we have rkDi(t,h) < n for each pair (t,h) G 
(T = Go(cxc). 

It remains to find a pair of matrices (X, Y) G g(l)xg(l) such that rk(X|Y) > n. Set 
X := (xij) where = 1 if i = j, and Xij = otherwise; Y := (7/^) where 7/jj = 1 
if 7 = j + m — 77 and y^ = otherwise. It is easy to see, that (X|Y) is a matrix of 
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the maximal possible rank, which equals min(m, 2n). In particular, rk(X|F) > n and 
(X,Y)£G (c(l)xc(l)). 



Example 2. Let g = so(V) = so 2n , where 2n = dimV. Consider a decomposition of V into 
a direct sum of two isotropic subspaces V = V + © V-. Suppose A G 0(V) and A\v + = id, 
A\y_ = —id. Then a conjugation by A defines an involution of g such that g = gl„ — 
gl(V + ) = £jt(VL). We have q 1 C Hom(V+,Vl) © Hom(F_, V+). More presicely, one can 
choose a basis in V + © V_ such that 

Go = Iff ,JL 1 | G GL n l , fl i = iY ° * 1 Gg[ n ,X = -X t ,F = -r* 



(fi*) -1 / J ' m IV Y 

Here g(l) = g x n Hom(V + , V_) = so n , g(-l) = fli H Kom(VL.,V+) = so n . For each pair 
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(£,?7) e fli x g! with f 
Cartan subspace c C gi consisting of skew symmetric anti-diagonal matrices, i.e., 



\{x oj l x = (^) ) ^ = 0if ^(^ + 1 -j)}- 



Now suppose n is odd and n = 2k + 1. When for each pair (i, ft) G cxc we have 
rkDi(t, ft) < n. Suppose g = ( ^ (B^)^ 1 ) G G °' If ^ = ( Y" 0^ ) G 01 ' then 
AdfoK = f .ntwvn-i V Therefore, g-D^^r]) := ^(AdfoK, Ad^) = 



n [B'y'YB- 1 

\ 

, ) is a non-degenerate 2nx2n matrix. Hence, 
OBJ 

TkDi(Ad(g)-£,Ad(g)-T]) = rk£>i(£, ?y) and rkDi(t, ft) < n for each pair (i, ft,) G Co- 
Let X, Z G g(l) be skew-symmetric nxn matrices of rank 2k such that the last column 
and the last row of X are zero, and the first row and the first column of Z are zero. Clearly, 
if k > 1, then (X\Z) has rank n = 2k + 1 and (A, F) £ G (c(l)xc(l)). Therefore, for 
(s02n,Q^n) with odd n > 3 condition (4) is not satisfied. 

Example 3. Consider now symmetric pair (i? 6 , sOio©k). Here g(l) = k l + and g(— 1) = k 16 

are different half-spinor representations of soio- Let t be a Cartan subalgebra of S0io, {vrj | 

% = 1, . . . , 5} fundamental weights of S0io, and {ei \ i = 1, . . . , 5} an orthogonal basis of 

t(R)* such that 7r 4 = (ei + £ 2 + £3 + £4 — £s)/2, 7r 5 = (ei + £ 2 + £3 + £4 + £s)/2 (for a detailed 

explanations of this notation see jSJ Reference Chapter]). The rank of (E 6 ,so w Q)k) equals 2. 

A Cartan subspace cCfli can be chosen such that c(l) is a t-invariant subspace with weights 

7r 5 and (^i — £ 2 — £ 3 — £4 — e 5 )/2. Let ft G t be an element such that £i(ft) = — 1, £i(ti) = 

for i = 2,..., 5. Clearly, \im te Q tt ^. +00 ex.p(th)-v = for each v G c(l)xc(l) C g(l)xg(l). 

It remains to find a non-trivial SOi -invariant in k[g(l)xg(l)] = l 5(k^ ( 6 ©k 16 ). Denote by 

V((p) a vector space of a representation with the highest weig ht (p. Then k™ = V(n 5 ) and 
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S 2 (k^) = l / (27T 5 ) © V(iti). We have the following SOio-invariant inclusions: 

^(^©kl 6 ) D S 2 (k^) ® <S 2 (ld 6 ) D V(7Ti) ® 7(7Ti). 

Since (l / (7r 1 )®V r (7r 1 )) s ° 10 = k, we get a required SOi -invariant / of degree 4. Therefore 
/(Go(c(l)xc(l)) = 0, and condition (4) is not satisfied. 
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